{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2c003380-b825-4c00-bb01-312f1ad4148f",
   "metadata": {},
   "source": [
    "# Model Overview\n",
    "For time series analysis, there are 5 maintream models falling in 3 different senarios:\n",
    "1. AR & MA & ARMA for stationary time series\n",
    "2. ARIMA for non-stationary time series without seasonal factors\n",
    "3. SARIMA for non-stationary time series with seasonal factors\n",
    "\n",
    "Among all aforementioned models, despite subtle differences with respect to their underlying mathmatical modeling, they share several standard processing steps."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75cb3fe1-6140-4392-b288-17e8a6c116b8",
   "metadata": {},
   "source": [
    "# Hypothesis of ARIMA\n",
    "1. mean of time series is one constant\n",
    "2. second moments of time series exists\n",
    "3. homogeneity of white noise variance\n",
    "4. white noise of each time point component is identicial to each other as well as other time point components"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a341e5b-b4cc-4c39-868b-e867df56bcc3",
   "metadata": {},
   "source": [
    "# stability & white noise test\n",
    "1. plot time-series graph\n",
    "2. use ADF for stability test\n",
    "3. use $Q_{LB}$ for white noise test\n",
    "\n",
    "## stability \n",
    "First, plot time series data with time as x-axis and values on y-axis:<br>\n",
    "1. plots with obvious tendency - non-stationary<br>\n",
    "2. plots with obivous periodicity - non-stationary<br>\n",
    "3. plots where series of values fluctuates around one certain line - stationary<br>\n",
    "if time series is non-stationary, do several k-order difference analysis on values, find out d value of ARIMA.<br>\n",
    "\n",
    "For stability test, use <font color=\"maroon\"><b>ADF(augmented Duckey-Fuller) test</font></b>:<br>\n",
    "Null hypothesis contends that all eigenroots are outside the unit circle, which implies that time series is not stable <br>\n",
    "If p value is smaller than $\\alpha$, we could safely state that time series is stable<br>\n",
    "\n",
    "python package <font color=\"maroon\"><b>statsmodels.tsa.stattools</font></b> function <font color=\"maroon\"><b>adfuller</font></b> encapsulates ADF process: <br>\n",
    "> parameter of adfuller\n",
    "```\n",
    "adfuller(\n",
    "    x,\n",
    "    maxlag: 'int | None' = None,\n",
    "    regression='c',\n",
    "    autolag='AIC',\n",
    "    store=False,\n",
    "    regresults=False,\n",
    ")\n",
    "\n",
    "x : array_like, 1d\n",
    "    The data series to test.\n",
    "maxlag : {None, int}\n",
    "    Maximum lag which is included in test, default value of\n",
    "    12*(nobs/100)^{1/4} is used when ``None``.\n",
    "regression : {\"c\",\"ct\",\"ctt\",\"n\"}\n",
    "    Constant and trend order to include in regression.\n",
    "\n",
    "    * \"c\" : constant only (default).\n",
    "    * \"ct\" : constant and trend.\n",
    "    * \"ctt\" : constant, and linear and quadratic trend.\n",
    "    * \"n\" : no constant, no trend.\n",
    "\n",
    "autolag : {\"AIC\", \"BIC\", \"t-stat\", None}\n",
    "    Method to use when automatically determining the lag length among the\n",
    "    values 0, 1, ..., maxlag.\n",
    "\n",
    "    * If \"AIC\" (default) or \"BIC\", then the number of lags is chosen\n",
    "      to minimize the corresponding information criterion.\n",
    "    * \"t-stat\" based choice of maxlag.  Starts with maxlag and drops a\n",
    "      lag until the t-statistic on the last lag length is significant\n",
    "      using a 5%-sized test.\n",
    "    * If None, then the number of included lags is set to maxlag.\n",
    "store : bool\n",
    "    If True, then a result instance is returned additionally to\n",
    "    the adf statistic. Default is False.\n",
    "regresults : bool, optional\n",
    "    If True, the full regression results are returned. Default is False.\n",
    "```\n",
    "\n",
    "> return values of adfuller:\n",
    "```\n",
    "adf : float\n",
    "    The test statistic.\n",
    "pvalue : float\n",
    "    MacKinnon's approximate p-value based on MacKinnon (1994, 2010).\n",
    "usedlag : int\n",
    "    The number of lags used.\n",
    "nobs : int\n",
    "    The number of observations used for the ADF regression and calculation\n",
    "    of the critical values.\n",
    "critical values : dict\n",
    "    Critical values for the test statistic at the 1 %, 5 %, and 10 %\n",
    "    levels. Based on MacKinnon (2010).\n",
    "icbest : float\n",
    "    The maximized information criterion if autolag is not None.\n",
    "resstore : ResultStore, optional\n",
    "    A dummy class with results attached as attributes.\n",
    "```\n",
    "\n",
    "## white noise\n",
    "For white noise test, we need to make sure that time seires we are interested in is not a mere random distribution.<br>\n",
    "Here we use <font color=\"maroon\"><b>LB(Ljung-Box) test</b></font>:<br>\n",
    "Null hyphthesis of LB test assumes that ACF of certain time series obey one certain normal distribution. <br>\n",
    "If p value is smaller than $\\alpha$, we could safely state that time series is not simple white noise<br>\n",
    "\n",
    "Python package <font color=\"maroon\"><b>statsmodels.stats.diagnostic</font></b> function <font color=\"maroon\"><b>acorr_ljungbox</font></b> encapsulates LB test process: <br>\n",
    "\n",
    "> parameters of LB test:\n",
    "```\n",
    "x : array_like\n",
    "    The data series. The data is demeaned before the test statistic is\n",
    "    computed.\n",
    "lags : {int, array_like}, default None\n",
    "    If lags is an integer then this is taken to be the largest lag\n",
    "    that is included, the test result is reported for all smaller lag\n",
    "    length. If lags is a list or array, then all lags are included up to\n",
    "    the largest lag in the list, however only the tests for the lags in\n",
    "    the list are reported. If lags is None, then the default maxlag is\n",
    "    min(10, nobs // 5). The default number of lags changes if period\n",
    "    is set.\n",
    "boxpierce : bool, default False\n",
    "    If true, then additional to the results of the Ljung-Box test also the\n",
    "    Box-Pierce test results are returned.\n",
    "model_df : int, default 0\n",
    "    Number of degrees of freedom consumed by the model. In an ARMA model,\n",
    "    this value is usually p+q where p is the AR order and q is the MA\n",
    "    order. This value is subtracted from the degrees-of-freedom used in\n",
    "    the test so that the adjusted dof for the statistics are\n",
    "    lags - model_df. If lags - model_df <= 0, then NaN is returned.\n",
    "period : int, default None\n",
    "    The period of a Seasonal time series.  Used to compute the max lag\n",
    "    for seasonal data which uses min(2*period, nobs // 5) if set. If None,\n",
    "    then the default rule is used to set the number of lags. When set, must\n",
    "    be >= 2.\n",
    "auto_lag : bool, default False\n",
    "    Flag indicating whether to automatically determine the optimal lag\n",
    "    length based on threshold of maximum correlation value.\n",
    "```\n",
    "\n",
    "> return values of LB test:\n",
    "```\n",
    "DataFrame\n",
    "    Frame with columns:\n",
    "\n",
    "    * lb_stat - The Ljung-Box test statistic.\n",
    "    * lb_pvalue - The p-value based on chi-square distribution. The\n",
    "      p-value is computed as 1 - chi2.cdf(lb_stat, dof) where dof is\n",
    "      lag - model_df. If lag - model_df <= 0, then NaN is returned for\n",
    "      the pvalue.\n",
    "    * bp_stat - The Box-Pierce test statistic.\n",
    "    * bp_pvalue - The p-value based for Box-Pierce test on chi-square\n",
    "      distribution. The p-value is computed as 1 - chi2.cdf(bp_stat, dof)\n",
    "      where dof is lag - model_df. If lag - model_df <= 0, then NaN is\n",
    "      returned for the pvalue.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8ccbceb-25f7-4cd2-892e-e2b3eaa71274",
   "metadata": {},
   "source": [
    "# Model selection\n",
    "1. plot PACF and ACF\n",
    "2. determine p,d,q values of ARIMA\n",
    "3. residue white noise test\n",
    "\n",
    "## Plot PACF & ACF\n",
    "1. for AR model, we could see trailing in ACF plots and truncation in PACF plots\n",
    "2. for MA model, we could see truncation in ACF plots and trailing in PACF plots\n",
    "3. for ARMA model, both ACF and PACF will show trailing effect in plots\n",
    "4. the highest ACF value which is higher than $2\\sigma$ of time seires meanwhile is closely next to bunch of values that are flunctuating between $\\pm 2\\sigma$ of time seires will be p value for AR model\n",
    "5. the highest PACF value which is higher than $2\\sigma$ of time seires meanwhile is closely next to bunch of values that are flunctuating between $\\pm 2\\sigma$ of time seires will be p value for MA model\n",
    "\n",
    "Python package <font color=\"maroon\"><b>statsmodels.tsa.stattools</b></font>  function <font color=\"maroon\"><b>acf, pacf</b></font> could be used to calculate acf and pacf values, latency is one mandatory parameter;<br>\n",
    "Python package <font color=\"maroon\"><b>statsmodels.graphics.tsaplots</b></font>  function <font color=\"maroon\"><b>plot_acf, plot_pacf</b></font> could be used to plot acf and pacf directly.<br>\n",
    "\n",
    "## ARMA & ARIMA modeling\n",
    "Python package <font color=\"maroon\"><b>statsmodels.tsa.arima.model</b></font> function <font color=\"maroon\"><b>ARIMA</font></b> encapsulates ARIMA process: <br>\n",
    "\n",
    "> parameters of AMIMA\n",
    "```\n",
    "endog : array_like, optional\n",
    "    The observed time-series process :math:`y`.\n",
    "exog : array_like, optional\n",
    "    Array of exogenous regressors.\n",
    "order : tuple, optional\n",
    "    The (p,d,q) order of the model for the autoregressive, differences, and\n",
    "    moving average components. d is always an integer, while p and q may\n",
    "    either be integers or lists of integers.\n",
    "seasonal_order : tuple, optional\n",
    "    The (P,D,Q,s) order of the seasonal component of the model for the\n",
    "    AR parameters, differences, MA parameters, and periodicity. Default\n",
    "    is (0, 0, 0, 0). D and s are always integers, while P and Q\n",
    "    may either be integers or lists of positive integers.\n",
    "trend : str{'n','c','t','ct'} or iterable, optional\n",
    "    Parameter controlling the deterministic trend. Can be specified as a\n",
    "    string where 'c' indicates a constant term, 't' indicates a\n",
    "    linear trend in time, and 'ct' includes both. Can also be specified as\n",
    "    an iterable defining a polynomial, as in `numpy.poly1d`, where\n",
    "    `[1,1,0,1]` would denote :math:`a + bt + ct^3`. Default is 'c' for\n",
    "    models without integration, and no trend for models with integration.\n",
    "    Note that all trend terms are included in the model as exogenous\n",
    "    regressors, which differs from how trends are included in ``SARIMAX``\n",
    "    models.  See the Notes section for a precise definition of the\n",
    "    treatment of trend terms.\n",
    "enforce_stationarity : bool, optional\n",
    "    Whether or not to require the autoregressive parameters to correspond\n",
    "    to a stationarity process.\n",
    "enforce_invertibility : bool, optional\n",
    "    Whether or not to require the moving average parameters to correspond\n",
    "    to an invertible process.\n",
    "concentrate_scale : bool, optional\n",
    "    Whether or not to concentrate the scale (variance of the error term)\n",
    "    out of the likelihood. This reduces the number of parameters by one.\n",
    "    This is only applicable when considering estimation by numerical\n",
    "    maximum likelihood.\n",
    "trend_offset : int, optional\n",
    "    The offset at which to start time trend values. Default is 1, so that\n",
    "    if `trend='t'` the trend is equal to 1, 2, ..., nobs. Typically is only\n",
    "    set when the model created by extending a previous dataset.\n",
    "dates : array_like of datetime, optional\n",
    "    If no index is given by `endog` or `exog`, an array-like object of\n",
    "    datetime objects can be provided.\n",
    "freq : str, optional\n",
    "    If no index is given by `endog` or `exog`, the frequency of the\n",
    "    time-series may be specified here as a Pandas offset or offset string.\n",
    "missing : str\n",
    "    Available options are 'none', 'drop', and 'raise'. If 'none', no nan\n",
    "    checking is done. If 'drop', any observations with nans are dropped.\n",
    "    If 'raise', an error is raised. Default is 'none'.\n",
    "```\n",
    "\n",
    "## Prediction of ARIMA\n",
    "1. after creating ARIMA model, use fit() function to invoke fitting process.\n",
    "2. to be slightly more specific, ARIMA will do MLE and parameter estimation and parameter hypothesis test, which will find their existance in results as prediction intervals and p values.\n",
    "1. model ARIMA contains prediction, using <font color=maroon><b>ARIMA.get_prediction(start=, end=)</b></font>\n",
    "2. prediction contains confidence interval, using <font color=maroon><b>prediction.conf_int(alpha=)</b></font>\n",
    "3. prediction contains predicted mean, using <font color=maroon><b>prediction.predicted_mean</b></font>\n",
    "\n",
    "## Residue white noise test\n",
    "Once arima model is created and fitted, its residue will be stored in <font color=red><b>resid</b></font> property, we can use this value for white noise test, as mentioned above, we could use LB-test for white noise test.<br>\n",
    "Yet for this time, we expect to see residue is random white noise, which means that all information is extracted by our ARIMA model, and the rest values are just random white noise.<br>\n",
    "Thus this time we want to see that p value is greater than $\\alpha$, which supports null hypothesis.\n",
    "\n",
    "# Python imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "6ba30231-a8e7-4a21-bce1-6ecd488b43bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "from statsmodels.tsa.stattools import acf, pacf\n",
    "from statsmodels.tsa.stattools import adfuller\n",
    "from statsmodels.tsa.arima.model import ARIMA\n",
    "from statsmodels.graphics.tsaplots import plot_acf, plot_pacf\n",
    "from statsmodels.stats.diagnostic import acorr_ljungbox"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec5842e7-3e9c-4e90-83ca-f3b55f180186",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
